Context Matters: An Analysis of assessments of XML Documents

The paper analyses searchers’ assessments of usefulness and specificity on different levels of granularity in XML-coded documents. Documents are assessed on 10 usefulness/specificity combinations and on the granularity levels of article, section, and subsection. Overlapping judgements show a remarkable lack of consistency between searchers. There is an inverse relationship between articles and sections both in the assessment of specificity and of usefulness, indicating that retrieval on different granularity levels are a useful feature of a retrieval system. Searchers find the full article more useful when they assess the same document both on the article and section level indicating that there is a need to provide context to the sections and subsections when presenting result list of XML-documents

Information science and cognitive psychology: a theoretical approach

Information, as a human and social phenomenon, is the object of study of an emergent scientific field named Information Science (IS), which we put forward as unitary and transdisciplinary and open to a rich interdisciplinarity with other fields of knowledge. In face of the new reality, baptized the Information Society', and the emergence of a new paradigm, that we name "post-custodial, scientific and informational", as opposed to the previous one, "historicist, custodial and technicist", it is urgent to consolidate the theoretical and methodological foundations of IS in order to develop research, both pure and applied, and to contribute to a definition of its boundaries as a scientific area, in the scope of Social Sciences. Starting from an operative definition of Information, this paper aims to discuss the cognitive and emotional dimension of the info-communicational phenomenon and, for that, it is crucial to start a profound and hard dialogue with Cognitive Sciences. The label of 'cognitivist' given, in IS literature, to some authors like Bertram Brookes, because of the emphasis he put on the passage from a state of knowledge to a new state through an addition of knowledge coming from an increase of information, sounds quite equivocal, because knowledge and cognition are not synonymous and cognitive and emotional activity is not reducible to formalities. It is necessary to compare concepts and to understand the neuropsychological roots of the production, the organization and the info-communicational behaviour, so the contribution of Neurosciences and Cognitive Sciences, namely Cognitive Psychology, is indispensable

Evaluating implicit feedback models using searcher simulations

In this article we describe an evaluation of relevance feedback (RF) algorithms using searcher simulations. Since these algorithms select additional terms for query modification based on inferences made from searcher interaction, not on relevance information searchers explicitly provide (as in traditional RF), we refer to them as implicit feedback models. We introduce six different models that base their decisions on the interactions of searchers and use different approaches to rank query modification terms. The aim of this article is to determine which of these models should be used to assist searchers in the systems we develop. To evaluate these models we used searcher simulations that afforded us more control over the experimental conditions than experiments with human subjects and allowed complex interaction to be modeled without the need for costly human experimentation. The simulation-based evaluation methodology measures how well the models learn the distribution of terms across relevant documents (i.e., learn what information is relevant) and how well they improve search effectiveness (i.e., create effective search queries). Our findings show that an implicit feedback model based on Jeffrey's rule of conditioning outperformed other models under investigation

Geographic information retrieval in a mobile environment: evaluating the needs of mobile individuals

This paper describes research that aims to define the information needs of mobile individuals, to implement a mobile information system that can satisfy those needs, and finally to evaluate the performance of that system with end-users. First a review of the emerging discipline of geographic information retrieval (GIR) is presented as background to the more specific issue of mobile information retrieval. Following this, a user needs study is described evaluating the requirements of potential users of a mobile information system; the study finds that there is a strong geographic component to users' information needs. Next, four geographic post-query filters are described which attempt to represent the region of space associated with an individual's query made at some specific spatial location. These filters are spatial proximity (distance in space), temporal proximity (travel time), speed-heading prediction surfaces (likelihood of visiting locations) and visibility (locations that can be seen). Two of these filters — spatial proximity and speed-heading prediction surfaces — are implemented in a mobile information system and subsequently evaluated with users in an outdoor setting. The results of evaluation suggest that retrieved information to which post-query geographic filters have been applied is considered more relevant than unfiltered information, and that users find information sorted by spatial proximity to be more relevant than that sorted by a prediction surface of likely future locations. The paper closes with a discussion of the wider implications of these results for developers of mobile information systems and location-based services

Improving a gold standard: treating human relevance judgments of MEDLINE document pairs

Given prior human judgments of the condition of an object it is possible to use these judgments to make a maximal likelihood estimate of what future human judgments of the condition of that object will be. However, if one has a reasonably large collection of similar objects and the prior human judgments of a number of judges regarding the condition of each object in the collection, then it is possible to make predictions of future human judgments for the whole collection that are superior to the simple maximal likelihood estimate for each object in isolation. This is possible because the multiple judgments over the collection allow an analysis to determine the relative value of a judge as compared with the other judges in the group and this value can be used to augment or diminish a particular judge’s influence in predicting future judgments. Here we study and compare five different methods for making such improved predictions and show that each is superior to simple maximal likelihood estimates

Active and passive utility of search interface features in different information seeking task stages

Models of information seeking, including Kuhlthau’s information Search Process model, describe fundamentally different macro-level stages. Current search systems usually do not provide support for these stages, but provide a static set of features predominantly focused on supporting micro-level search interactions. This paper investigates the utility of search user interface (SUI) features at different macro-level stages of complex tasks. A user study was designed, using simulated work tasks, to explicitly place users within different stages of a complex task: pre-focus, focus, and post-focus. Active use, passive use and perceived usefulness of features were analysed in order to derive when search features are most useful. Our results identify significant differences in the utility of SUI features between each stage. Specifically, we have observed that informational features are naturally useful in every stage, while input, control features decline in usefulness after the pre-focus stage, and personalisable features become more useful after the pre-focus stage. From these findings, we conclude that features less commonly found in web search interfaces can provide value for users, without cluttering simple searches, when provided at the right times

Machine Learning in Automated Text Categorization

The automated categorization (or classification) of texts into predefined categories has witnessed a booming interest in the last ten years, due to the increased availability of documents in digital form and the ensuing need to organize them. In the research community the dominant approach to this problem is based on machine learning techniques: a general inductive process automatically builds a classifier by learning, from a set of preclassified documents, the characteristics of the categories. The advantages of this approach over the knowledge engineering approach (consisting in the manual definition of a classifier by domain experts) are a very good effectiveness, considerable savings in terms of expert manpower, and straightforward portability to different domains. This survey discusses the main approaches to text categorization that fall within the machine learning paradigm. We will discuss in detail issues pertaining to three different problems, namely document representation, classifier construction, and classifier evaluation.Comment: Accepted for publication on ACM Computing Survey